Imaging device and control method

ABSTRACT

An imaging device including an imaging element configured to output an image pickup signal and a parallax image signal is provided. The imaging device selects a subject area on the basis of the image pickup signal and performs a control to read the parallax image signal from a reading area of the imaging element corresponding to the selected subject area. In this control, the imaging device selects a subject area on the basis of a result of comparing the number of lines configuring the reading area corresponding to the subject area and a threshold value.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an imaging device and a control method.

Description of the Related Art

As the number of pixels in television broadcasting has recentlyincreased, an imaging device in which the number of pixels has increasedto 4K or 8K has been proposed. In addition, an imaging device in whichthere has been progress to a high frame rate to display a subject in asmooth manner has also been proposed.

As the resolution of imaging devices has increased, an improvement offocusing precision of a lens and speeding-up of a focusing operationhave been required. Therefore, in order to realize speeding-up of afocusing operation and improvement of focusing precision of an imagingdevice, an imaging device which includes an imaging element capable ofacquiring a parallax image signal, that is, an image signal foracquiring phase difference information, and an image pickup signal atthe same time has been proposed. Japanese Patent Laid-Open No.2014-137567 discloses an imaging device which includes an imagingelement having a focus detection pixel configured to output a signalused for calculation of a defocus amount and an imaging pixel.

The imaging device disclosed in Japanese Patent Laid-Open No.2014-137567 reads signals from the focus detection pixel and the imagingpixel at the same time, and thus has an increased reading band of asignal from the imaging element. If the reading band of a signal fromthe imaging element (the number of vertical lines) exceeds the referencenumber of lines, reading malfunction occurs. In addition, if the numberof terminals for outputting a signal increases according to an increasein the reading band, a package of the imaging element becomes larger andpower consumption increases.

SUMMARY OF THE INVENTION

The present invention provides an imaging device which includes animaging element configured to output a parallax image signal and animage pickup signal and is capable of preventing reading malfunctioncaused by an excess of the reading band of signals from the imagingelement.

An imaging device according to a first embodiment of the presentinvention includes an imaging element configured to output an imagepickup signal and a parallax image signal, and a control unit configuredto select a subject area on the basis of the image pickup signal and toperform a control to read the parallax image signal from a reading areaof the imaging element corresponding to the selected subject area. Thecontrol unit selects a subject area on the basis of a result ofcomparing the number of lines configuring the reading area correspondingto the subject area and a threshold value.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram which shows a configuration example of an imagingdevice according to the present embodiment.

FIGS. 2A and 2B are diagrams which show an example of a configuration ofan imaging element.

FIGS. 3A and 3B are diagrams which show a setting of an image area foracquiring phase difference information.

FIGS. 4A and 4B are diagrams which show the setting of an image area foracquiring phase difference information.

FIG. 5 is a diagram which shows a relationship between an in-focusposition and a focal position of a face detection frame.

FIGS. 6A and 6B are diagrams which describe the setting of an image areafor acquiring phase difference information in the first embodiment.

FIG. 7 is a flowchart which describes processing of setting an imagearea for acquiring phase difference information.

FIG. 8 is a flowchart which describes the processing of setting an imagearea for acquiring phase difference information.

FIG. 9 is a diagram which describes a reading time division control ofan image area for acquiring phase difference information in a secondembodiment.

FIG. 10 is a diagram which describes a method for calculating focalpositions of a first selection frame and other frames of an image foracquiring phase difference information.

FIG. 11 is a diagram which describes a frame rate of a subjectphotographic image in a third embodiment and a threshold value of animage for acquiring phase difference information.

FIG. 12 is a diagram which shows a configuration example of a CPUprovided in the imaging device.

DESCRIPTION OF THE EMBODIMENTS

An imaging device which specifies a reading area of an imaging elementcorresponding to a subject (for example, face) detected in aphotographed image and reads a parallax image signal from the readingarea will be considered. However, if there are a plurality of areas ofthe detected subject (subject areas), there is a possibility that areading band may increase and reading malfunction may occur.

First Embodiment

FIG. 1 is a diagram which shows a configuration example of an imagingdevice according to the present embodiment.

An imaging device shown in FIG. 1 includes a lens 1 to a CPU 7.

The lens 1 configures an imaging unit for imaging a subject. An imagingelement 2 converts an optical signal from the lens 1 into an electricalsignal. The imaging element 2 has pixels which are capable of reading animage area for acquiring a phase difference and arranged on X-Yaddresses. The image area for acquiring a phase difference is a readingarea for reading a parallax image to be described below.

The imaging element 2 is configured from one microlens and twophotodiodes for a color filter. A signal (image pickup signal) obtainedby adding signals from the two photodiodes and an output signal(parallax image signal) from one of the two photodiodes are output fromthe imaging element 2 at the same time. In the following description, animage related to the image pickup signal is described as a capturedimage, and an image related to the parallax image signal is described asa parallax image.

A development processing unit 3 develops a captured image output fromthe imaging element 2. A face detection unit 5 extracts a face area fromthe developed captured image. A face detection line measurement unit 6measures the number of vertical lines in the extracted face area on thebasis of an output signal from the face detection unit 5.

A central processing unit (CPU) 7 controls an entire imaging device. Theoutput signal from the face detection unit 5 and an output signal fromthe face detection line measurement unit 6 are input to the CPU 7. TheCPU 7 sets a priority of a face detection frame on the basis of areference number of lines and the output signal from the face detectionunit 5. The reference number of lines is a number of vertical lineswhich is a predetermined reference. The CPU 7 sets an image area foracquiring phase difference information according to the prioritized facedetection frame.

Here, an internal operation of the CPU 7 will be described withreference to FIG. 12. FIG. 12 is a diagram which shows a configurationexample of a CPU provided in the imaging device.

The output signal from the face detection unit 5 and the output signalfrom the face detection line measurement unit 6 are input to the CPU 7.The output signal from the face detection line measurement unit 6 isinput to a comparator 101. On the other hand, information on a thresholdvalue for a total number of lines of an image for acquiring phasedifference information from the imaging element 2 (the reference numberof lines) is input to the comparator 101 according to the predeterminedreference number of lines 100. The comparator 101 determines whether thetotal number of lines of an image for acquiring phase differenceinformation corresponding to a current face detection frame is equal toor greater than, or equal to or less than the reference number of lines,and outputs information of a frame selection 102 based on the two inputsignals.

Frame information from the face detection unit 5 is input to the frameselection 102. Moreover, information on a signal output from a phasedifference information calculation unit 4 via a defocus amountcalculation 104 for calculating a defocus amount of each face detectionframe is input to the frame selection 102. Furthermore, information of aframe set as a first selection frame among a plurality of detected facedetection frames is also input to the frame selection 102. The frameselection 102 outputs priority information for the plurality of detectedface detection frames and area information of an image for acquiringphase difference information corresponding to a face detection frame tobe read within one frame to a phase difference image reading area setvalue calculation unit 103 among the four types of information whichhave been input. The phase difference image reading area set valuecalculation unit 103 calculates a reading set value for the imagingelement 2 and outputs it to the imaging element 2.

The phase difference information calculation unit 4 generates the otherparallax image (R) based on a captured image (R+L) and a parallax image(L). The phase difference information calculation unit 4 calculatesphase difference information on the basis of the parallax image (L) andthe parallax image (R).

FIG. 2 is a diagram which shows an example of a configuration of theimaging element.

The imaging element 2 includes, as shown in FIGS. 2A and 2B, onemicrolens, and two photodiodes for a color filter which configure onepixel. The pixel is arranged on an X-Y address. The imaging element 2configured from each of pixels outputs a captured image obtained byadding output signals from the two photodiodes and a parallax image Lthat is an output signal from one of the two photodiodes as a RAW imageat the same time. The imaging element 2 outputs the captured image tothe development processing unit 3. In addition, the imaging element 2outputs the captured image and the parallax image to the phasedifference information calculation unit 4.

The development processing unit 3 performs development processing suchas interpolation processing and color correction processing of eachpixel on the RAW image that is the input captured image to generate adeveloped image (YUV image). The development processing unit 3 outputsthe developed image to the face detection unit 5.

The phase difference information calculation unit 4 generates a parallaximage (R) by subtracting the parallax image (L) from the input capturedimage. The phase difference information calculation unit 4 calculatesphase difference information by comparing the parallax image (L) and theparallax image (R). The calculated phase difference information can beused as, for example, a focusing signal, and focusing can be performedby transmitting a control signal to a focusing unit provided in the lens1.

The face detection unit 5 determines whether there is a face in the YUVimage input from the development processing unit 3, and if there is aface, the face detection unit 5 sets an area corresponding to the faceas a face area and outputs the face area to the face detection linemeasurement unit 6. In addition, the face detection unit 5 outputspositional information of the face area in a vertical direction to theCPU 7.

The face detection line measurement unit 6 measures the number ofvertical lines Lf from the input information of the face area. The facedetection line measurement unit 6 outputs information on the number ofvertical lines Lf to the CPU 7. In addition, if there are a plurality offace areas detected by the face detection unit 5, the face detectionunit 5 outputs information on all the face detection frames to the facedetection line measurement unit 6. Then, the face detection linemeasurement unit 6 outputs information on the number of vertical linesLf (1 to N) of areas corresponding to respective face detection framesto the CPU 7.

The CPU 7 controls the entire imaging device. The positional informationof the face area in a vertical direction is input to the CPU 7 from theface detection unit 5. In addition, the information on the number ofvertical lines Lf (1 to N) of the plurality of face areas is input tothe CPU 7 from the face detection line measurement unit 6. The CPU 7calculates a total value LfS of Lf (1 to N). Moreover, the CPU 7includes the reference number of lines Lc for comparison with the totalnumber of vertical lines LfS of face areas.

In addition, the CPU 7 sets a main face as a first selection frame whichis a selection frame with a highest priority among the plurality ofinput face areas (positions and sizes of faces), and calculates a focalposition of the first selection frame and focal positions of otherframes. The CPU 7 sets a face area whose focal position is closest tothe first selection frame as a second selection frame which is aselection frame with a second priority, and sets a face area whose focalposition is next closest as a third selection frame which is a selectionframe with a third priority. The main face can also be arbitrarily setby a user.

Here, a method of calculating a focal position of each frame will bedescribed with reference to FIG. 10. FIG. 10 is a diagram whichdescribes a method for calculating a focal position of the firstselection frame and focal positions of other frames of an image foracquiring phase difference information.

Reading images for acquiring phase difference information of respectiveframes from the imaging element 2 is performed by switching betweenimage areas for acquiring phase difference information of respectiveframes at each time to read frames A to D in a time division manner. InFIG. 10, the maximum number of lines for reading an image area foracquiring phase difference information from the imaging element 2 is setto 700, and areas corresponding to respective face detection framesinclude, as shown in FIG. 4, 450 for a face detection frame A, 330 for aface detection frame B, and 220 for a face detection frame C.

In addition, the first selection frame is set as the face detectionframe A.

For this reason, firstly in a time of the frame A, A frame and B frameare read as an image area for acquiring phase difference information.

Next, in a time of the frame B, A frame and C frame are read as an imagearea for acquiring phase difference information.

Next, in a time of the frame C, A frame and B frame are read as an imagearea for acquiring phase difference information.

Next, in a time of the frame D, A frame and C frame are read as an imagearea for acquiring phase difference information.

Next, a phase difference of each frame is calculated based on each imagearea for acquiring phase difference information which has been read ineach frame to calculate a defocus amount of each frame.

Then, frames having focal positions close to the first selection frame Aare determined and are set as a second selection frame and a thirdselection frame based on a result of the defocus amount calculation.

The CPU 7 compares the total number of vertical lines LfS of face areasand the reference number of lines Lc which is a threshold value, andselects a face area (subject area) corresponding to an image area foracquiring phase difference information on the basis of a result of thecomparison. Specifically, the CPU 7 executes the following processing ifthe total number of vertical lines LfS of face areas is equal to or lessthan the reference number of lines Lc (LfS≦Lc). The CPU 7 selects allface areas that have been input, and sets areas corresponding to theface areas as image areas for acquiring phase difference information inthe imaging element 2.

In addition, the CPU 7 executes the following processing if the totalnumber of vertical lines of face areas LfS is greater than the referencenumber of lines Lc (LfS>Lc). The CPU 7 sets an area corresponding to thefirst selection frame as an image area for acquiring phase differenceinformation.

In addition, the CPU 7 executes the following processing if the numberof vertical lines Lf1 of the first selection frame is equal to or lessthan the reference number of lines Lc (Lf1≦Lc). The CPU 7 sets thesecond selection frame as a selection candidate. Moreover, if a total ofthe number of vertical lines Lf1 of the first selection frame and thenumber of vertical lines Lf2 of the second selection frame is equal toor less than the reference number of lines Lc, the CPU 7 executes thefollowing processing. The CPU 7 selects the first selection frame andthe second selection frame, and sets areas corresponding to theseselection frames as an image reading area for acquiring phase differenceinformation.

Furthermore, if a total of the number of vertical lines Lf1 of the firstselection frame, the number of vertical lines Lf2 of the secondselection frame, and the number of vertical lines Lf3 of the thirdselection frame is equal to or less than the reference number of linesLc, the CPU 7 executes the following processing. The CPU 7 sets theimage reading area for acquiring phase difference information to includean area corresponding to the third selection frame. As described above,the CPU 7 sets an area corresponding to a selection frame as the imagereading area for acquiring phase difference information in accordancewith priority so that the total number of vertical lines of selectionframes does not exceed the reference number of lines Lc.

FIGS. 3 and 4 are diagrams which describe a setting of an image area foracquiring phase difference information.

FIG. 3A shows a captured image. An image size of the captured image is4K*2K (4096 pixels*2160 lines). A face detection frame A is shown in thecaptured image shown in FIG. 3A. The reference number of lines Lcprovided in the CPU 7 is set to 700 lines.

In an example shown in FIG. 3A, an area corresponding to a facedetection frame A is a candidate set as an image area for acquiringphase difference information A. FIG. 3B is a diagram which shows theimage area for acquiring phase difference information A. Information onthe number of vertical lines Lf of the image area for acquiring phasedifference information A is 450 lines.

In examples shown in FIGS. 3A and 3B, since there is one face detectionframe and LfS is 450, LfS is less than Lc (Lfs<Lc). As a result, theimage area for acquiring phase difference information set in the imagingelement 2 is only the image area for acquiring phase differenceinformation A corresponding to the face detection frame A.

In an example shown in FIG. 4A, there are three face detection frameswhich are face detection frames A, B, and C in a photographed image. Inthis example, as shown in FIG. 4B, image areas for acquiring phasedifference information A, B, and C corresponding to each of the facedetection frames A, B, and C are candidates for the image area foracquiring phase difference information which is set. Information LfA,LfB, and LfC on the number of vertical lines of each of the image areasfor acquiring phase difference information A, B, and C are 450 lines,220 lines, and 330 lines, respectively.

In examples shown in FIGS. 4A and 4B, since there are three facedetection frames, LfS which is obtained by adding LfA, LfB, and LfC is1000. As a result, LfS is greater than Lc (LfS>Lc) and exceeds areference value. For this reason, it is not possible to read all theimage areas for acquiring phase difference information from the imagingelement 2.

FIG. 5 is a diagram which shows relationships between in-focus positionsand focal positions of a plurality of face detection frames shown inFIG. 4.

The face detection frame A, the face detection frame B, and the facedetection frame C are arranged in this order from the closest focalposition toward an infinity focal position. It is known that a focalposition of the face detection frame A is the closest to a focalposition of the face detection frame B in FIG. 5.

FIG. 6 is a diagram which describes the setting of the image area foracquiring phase difference information in the first embodiment.

As shown in FIG. 4B, there are three face detection frames in thephotographed image and the reference number of lines Lc is set to 700.FIG. 6A shows the setting of the image area for acquiring phasedifference information if the face detection frame A is specified as thefirst selection frame which is a selection frame with a highest priorityamong the three face detection frames.

The information on the number of vertical lines Lf corresponding to theface detection frame A (=LfA) is 450, and the reference number of linesLc is 700. Therefore, if the vertical lines corresponding to the facedetection frame A are set as the image area for acquiring phasedifference information and read, there is still a portion of 250 linesleft over with respect to the reference number of lines Lc.

Therefore, the face detection frame B which is a frame whose focalposition is the closest to a focal position of the first selectionframe, that is, the face detection frame A, is set as a candidate forthe second selection frame. In this case, the information on the numberof vertical lines is in a following state.

The information on the number of vertical lines Lf=LfA+LfB=450+220=670

The reference number of lines Lc=700

In this state, even if the vertical lines corresponding to the facedetection frame A and the vertical lines corresponding to the facedetection frame B are set as the image area for acquiring phasedifference information and read with respect to the reference number oflines Lc, there is still a portion of 30 vertical lines left over withrespect to the reference number of lines Lc.

Next, the face detection frame C which is a frame whose focal positionis closest to the focal position of the face detection frame A next tothe face detection frame B is set as a candidate for the third selectionframe. In this case, information on the number of vertical lines is in afollowing state.

Information on the number of vertical linesLf=LfA+LfB+LfC=450+220+330=1000

The reference number of lines Lc=700

In this state, reading up to the third selection frame causes the totalnumber of vertical lines Lf to exceed the reference number of lines Lc.Therefore, if the first selection frame is set as the face detectionframe A, the image area for acquiring phase difference information readfrom the imaging element 2 is the image area for acquiring phasedifference information corresponding to the face detection frame A andthe face detection frame B.

In an example shown in FIG. 6B, it may be assumed that the facedetection frame B is specified as the first selection frame among thethree face detection frames.

When the face detection frame B is selected as the first selectionframe, the information on the number of vertical lines Lf=LfB is 220. Inaddition, the reference number of lines Lc is 700.

In this state, if the vertical lines corresponding to the face detectionframe B are set as the image area for acquiring phase differenceinformation and read, there is still a portion of 480 vertical linesleft over with respect to the reference number of lines Lc. Therefore,the face detection frame C which is a frame whose focal position is theclosest to a focal position of the first selection frame, that is, theface detection frame B, is set as a candidate for the second selectionframe. In this case, information on the number of vertical lines is in afollowing state.

The information on the number of vertical lines Lf=LfB+LfC=220+330=550

The reference number of lines Lc=700

In this state, even if the vertical lines corresponding to the facedetection frame B and the vertical lines corresponding to the facedetection frame C are set as the image area for acquiring phasedifference information and read, there is still a portion of 150vertical lines left over with respect to the reference number of linesLc.

Next, if the face detection frame C is set as a candidate for the thirdselection frame, the number of vertical lines of the face detectionframe A is 450 lines, and thus reading up to the third selection framecauses the total number of vertical lines to exceed the reference numberof lines Lc. Therefore, if the first selection frame is set as the facedetection frame B, the image area for acquiring phase differenceinformation read from the imaging element 2 is an image area foracquiring phase difference information corresponding to the facedetection frame B and the face detection frame C. As described above, ifthere are a plurality of face detection frames in a photographed image,the reference number of lines set in advance is compared with the numberof vertical lines of the first selection frame, and if there is asurplus, the number of candidates for set frames such as the secondselection frame and the third selection frame is increased. Then, theimage area for acquiring phase difference information is set so that atotal of the number of vertical lines of the candidates for set framesdoes not exceed the reference number of lines. That is, the CPU 7 sets aface detection frame of a main subject to have a highest priority andsets a face detection frame closer to a focal position of the mainsubject to have a higher priority. Then, face detection framescorresponding to the image area for acquiring phase differenceinformation are selected in a descending order of priority so that thetotal number of vertical lines corresponding to a selected subject areadoes not exceed the reference number of lines Lc.

FIGS. 7 and 8 are flowcharts which describe processing of setting theimage area for acquiring phase difference information.

In S1000, the CPU 7 starts reading from the imaging element 2 andoutputs an image for acquiring phase difference information and aphotographed image at the same time. In S1001, the face detection unit 5performs extraction of a face area (face detection) from a YUV imageobtained by performing development processing on a RAW image for thephotographed image output from the imaging element 2.

Next, in S1002, the CPU 7 determines whether there is a face in thephotographed image based on a result of the face detection. If there isno face in the photographed image, processing proceeds to S1003. InS1003, the CPU 7 specifies a position for reading a reference phasedifference stored in a storage unit in advance as an image area foracquiring phase difference information. Then, the processing proceeds toS1016 of FIG. 8 and the CPU 7 sets the image area for acquiring phasedifference information in the imaging element 2.

If there is a face in the photographed image, the CPU 7 sets a frame ofthe detected face (face detection frame). The processing proceeds toS1004. In S1004, the face detection line measurement unit 6 measures thenumber of vertical lines corresponding to respective face detectionframes. Subsequently, in S1005, the CPU 7 calculates a total of thenumbers of vertical lines corresponding to respective face detectionframes, which have been measured. In S1006, the CPU 7 compares the totalof the number of vertical lines with the reference number of linesstored in the storage unit in advance.

In S1007, the CPU 7 determines whether the total of the number ofvertical lines exceeds the reference number of lines. If the total ofthe number of vertical lines does not exceed the reference number oflines, the processing proceeds to S1017 of FIG. 8. Then, in S1017, theCPU 7 specifies an area of vertical lines corresponding to all theextracted face detection frames as an image area for acquiring phasedifference information, and the processing proceeds to S1016. Then, inS1016, the CPU 7 sets data in a timing generator provided in the imagingelement 2 to read an image from the image area for acquiring phasedifference information set in the imaging element.

If the total of the number of vertical lines exceeds the referencenumber of lines, the processing proceeds to S1008. In S1008, the CPU 7detects which face detection frame is set as a main face among the facedetection frames. The CPU 7 sets the face detection frame set as a mainface as the first selection frame. Subsequently, in S1009, the CPU 7calculates focal positions of respective face detection frames andcompares the focal positions of the first selection frame and otherselection frames. In S1010, the CPU 7 sets a second selection frame anda third selection frame among the face detection frames other than thefirst selection frame on the basis of a result of the comparisonprocessing in S1009.

In S1011 of FIG. 8, the CPU 7 calculates the number of vertical linescorresponding to the first selection frame and the second selectionframe which have been set and calculates a total of the number ofvertical lines. Subsequently, in S1006, the CPU 7 compares the total ofthe number of vertical lines calculated in S1011 with the referencenumber of lines.

Next, in S1007, the CPU 7 determines whether the total of the number ofvertical lines exceeds the reference number of lines on the basis of aresult of the comparison in S1016. If the total of the number ofvertical lines exceeds the reference number of lines, the processingproceeds to S1012. In S1012, the CPU 7 specifies an area correspondingto the first selection frame as an image area for acquiring phasedifference information. Then, in S1016, the CPU 7 sets data in thetiming generator provided in the imaging element.

If the total of the number of vertical lines does not exceed thereference number of lines, the processing proceeds to S1013. In S1013,the CPU 7 calculates a total of the number of vertical lines of thefirst selection frame, the second selection frame, and the thirdselection frame. In S1006, the CPU 7 compares the total of the number ofvertical lines with the reference number of lines.

Next, the CPU 7 determines whether the total of the number of verticallines exceeds the reference number of lines. If the total of the numberof vertical lines exceeds the reference number of lines, the processingproceeds to S1014. In S1014, the CPU 7 specifies an area correspondingto the first selection frame and the second selection frame as an imagearea for acquiring phase difference information. Then, in S1016, the CPU7 sets data in the timing generator provided in the imaging element.

If the total of the number of vertical lines does not exceed thereference number of lines, the processing proceeds to S1015. In S1015,the CPU 7 specifies an area corresponding to each of the first selectionframe, the second selection frame, and the third selection frame as animage area for acquiring phase difference information. Then, in S1016,the CPU 7 sets data in the timing generator provided in the imagingelement. By the above processing, even if there are a plurality of facedetection frames in a photographed image, it is possible to controlwhich frame to specify as an image area for acquiring phase differenceinformation according to priority settings of the face detection frames.Accordingly, it is possible to prevent poor reading due to an excess ofreading band of an imaging element.

Second Embodiment

The imaging device of the second embodiment reads a plurality of imageareas for acquiring phase difference information in conjunction with theplurality of face detection frames in the photographed image in a timedivision manner.

FIG. 9 is a diagram which describes a reading time division control ofthe image area for acquiring phase difference information in the secondembodiment.

The reference number of lines Lc is set to 500 and described. Thereference number of lines of an image area for acquiring phasedifference information corresponding to each of three face detectionframes shown in FIG. 4B is as follows.

Face detection frame A: 450

Face detection frame B: 220

Face detection frame C: 330

Since the reference number of lines Lc is 500, it is not possible to setan image area for acquiring phase difference information correspondingto all the face detection frames to a reading target area in one frame.

In an example shown in FIG. 9, if the face detection frame A of thethree face detection frames is specified as a first selection frame, theCPU 7 sets the face detection frame B whose focal position is theclosest to the face detection frame A as a second selection frame. Areading frame rate of the imaging element 2 is set to 120 fps (outputimages of 120 frames per second).

The CPU 7 sets reading of the first selection frame and the secondselection frame in a time division manner. Specifically, the CPU 7 setsa reading time rate between the first selection frame and the secondselection frame to 2:1. That is, the CPU 7 controls such that the secondselection frame is read for one frame after reading the first selectionframe for two frames. With respect to the reading frame rate of 120 fpsof the imaging element 2, there is 80 fps for an image area foracquiring phase difference information corresponding to the facedetection frame A, and 40 fps for an image area for acquiring phasedifference information corresponding to the face detection frame B. As aresult, even if a plurality of face detection frames are present in asubject photographic image and only an image area for acquiring phasedifference information of one face detection frame can be read, it ispossible to read an image area for acquiring phase differenceinformation corresponding to the plurality of face detection frames. Areading time rate between the plurality of face detection frames may beset to an arbitrary value. In addition, the reading time rate may bechanged according to a reading frame rate of an imaging device and thenumber of reading pixels of an imaging element.

In the present embodiment, the CPU 7 determines whether a plurality ofsubject areas can be selected in one frame on the basis of a result ofcomparing the number of vertical lines corresponding to a face detectionframe and the reference number of lines. If a plurality of subject areascannot be selected in one frame, the CPU 7 performs a control to read animage area for acquiring phase difference information corresponding to aface detection frame of a first subject and an image area for acquiringphase difference information corresponding to a face detection frame ofa second subject in a time division manner. The CPU 7 sets a secondframe rate to read a parallax image signal from the image area foracquiring phase difference information corresponding to the facedetection frame of a second subject to be lower than a first frame rateto read a parallax image signal from the image area for acquiring phasedifference information corresponding to the face detection frame of afirst subject.

In addition, as a priority of the plurality of face detection frames, apriority set in advance may be used, and a priority of the facedetection frames may be determined according to a user's specification.That is, the CPU 7 may set a main subject as a first subject and set asubject whose focal position is the closest to the main subject or asubject specified according to a user's operation as a second subject.According to the imaging device of the second embodiment, it is possibleto prevent poor reading due to an excess of reading band of an imagingelement.

Third Embodiment

In an imaging device of a third embodiment, a frame rate of a subjectphotographic image and a method of setting a threshold value of readinglines of an image for acquiring a phase difference at this time will bedescribed with reference to FIG. 11.

FIG. 11 shows a frame rate of a subject photographic image and athreshold value of reading lines of an image for acquiring phasedifference information when the frame rate is selected.

Here, Equation (1) for calculating the threshold value is shown.

(H*LAB+H*LA)*F=X  (1)

The number of horizontal pixels of an image for photographing a subject(A+B) and an image for acquiring a phase difference (A): H

The number of vertical lines of the image for photographing a subject(A+B): LAB

The number of vertical lines of the image for acquiring a phasedifference (A): LA

A frame rate of the image for photographing a subject (A+B): F

A maximum reading rate of an imaging element: X

Since the calculation expression is set as described above, for example,if LAB is 2160 lines and X is 142560,

if the frame rate F is 24 fps, LA is 3780, and

if the frame rate F is 60 fps, LA is 216.

Therefore, since the number of vertical lines of the image forphotographing a subject is 2160 lines, the total number of lines (2160lines) if the frame rate is set to 24 fps or 216 lines which is 10% ofthe total number of lines if the frame rate is set to 60 fps is set as athreshold value of reading lines of the image for acquiring a phasedifference.

As described above, it is possible to change a threshold value of thenumber of vertical lines corresponding to a reading area of the imagefor acquiring a phase difference according to a setting of a frame rateof the image for photographing a subject of an imaging device or amultiplied value of the number of horizontal pixels of the image forphotographing a subject, that is, the number of pixels in one frame.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment (s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment (s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-099186, filed May 18, 2016, Japanese Patent Application No.2017-036568, filed Feb. 28, 2017, which are hereby incorporated byreference wherein in their entirety.

What is claimed is:
 1. An imaging device comprising: an imaging elementconfigured to output an image pickup signal and a parallax image signal;and a control unit configured to select a subject area on the basis ofthe image pickup signal and to perform a control to read the parallaximage signal from a reading area of the imaging element corresponding tothe selected subject area, wherein the control unit selects a subjectarea on the basis of a result of comparing the number of linesconfiguring the reading area corresponding to the subject area and athreshold value.
 2. The imaging device according to claim 1, wherein, ifthere are a plurality of subject areas, the control unit selects thesubject area so that the number of lines corresponding to the readingarea does not exceed the threshold value.
 3. The imaging deviceaccording to claim 2, wherein the control unit determines a priority onthe basis of a focal position corresponding to a subject area andselects the subject area on the basis of the determined priority.
 4. Theimaging device according to claim 3, wherein the control unit sets asubject area of a main subject to have a highest priority, sets asubject area of a subject whose focal position is closer to a focalposition of the main subject to have a higher priority, and selectssubject areas in a descending order of priority so that a total of thenumber of lines configuring the reading area corresponding to theselected subject areas does not exceed the threshold value.
 5. Theimaging device according to claim 1, wherein the control unit performs acontrol to read a reading area corresponding to a subject area of afirst subject and a reading area corresponding to a subject area of asecond subject in a time division manner.
 6. The imaging deviceaccording to claim 5, wherein the control unit determines whether aplurality of subject areas can be selected in one frame on the basis ofa result of comparing the number of lines configuring the reading areacorresponding to a subject area and a threshold value, and if aplurality of subject areas cannot be selected in one frame, the controlunit performs a control to read the reading area corresponding to asubject area of a first subject and the reading area corresponding to asubject area of a second subject in a time division manner.
 7. Theimaging device according to claim 5, wherein the control unit sets asecond frame rate to read the parallax image signal from the readingarea corresponding to a subject area of a second subject to be lowerthan a first frame rate to read the parallax image signal from thereading area corresponding to a subject area of a first subject.
 8. Theimaging device according to claim 5, wherein the control unit sets amain subject as the first subject, and sets a subject whose focalposition is the closest to a focal position of the main subject or asubject specified according to a user's operation as the second subject.9. A method of controlling an imaging device which includes an imagingelement for outputting an image pickup signal and a parallax imagesignal, the method comprising: controlling which selects a subject areaon the basis of the image pickup signal, and performs a control to readthe parallax image signal from a reading area of the imaging elementcorresponding to the selected subject area, wherein a subject area isselected on the basis of a result of comparing the number of linesconfiguring the reading area corresponding to the subject area and athreshold value in the controlling.